A Comparative Assessment of Machine Learning Classification Methods For The Detection of Wilt Disease in Pine and Oak Trees

by Tem Cavanagh as part of STAT6020 Predictive Analytics (Master of Data Science)

---

Abstract: This paper examines the classification accuracy of machine learning methods in detecting wilt disease present in Japanese pine and oak trees. A comparative assessment is conducted on the wilt dataset (UCI Machine Learning Repository) whereby the classification accuracy of Decision Trees (Random Forests) and Support Vector Machines are analysed. Formal quantitative assessment of results are carried out in combination with qualitative assessment through visualisation of results. FINDINGS. CONCLUSIONS.

---

Introduction:

Forests are becoming increasingly susceptible to damage and degradation by way of plant diseases caused by insects and climate change (Food and Agriculture Organization of the United Nations Regional Office for Asia and the Pacific 2010; Garrett et. al. 2006).

Discolouration of foliage is often an indicator of a diseased tree (Johnson). Johnson et. al. have conducted high resolution satellite image surveys with the purpose of identifying diseased pine and oak trees in Japan. The resulting images are shown below:

Figure 1: Diseased classified trees (Source: Johnson)

The objective of this project is to assess and compare the classification accuracy of statistical learning methods in detecting and classifying diseased trees. The overarching aim of this project is to demonstrate the applicability of statistical learning techniques as preventative measures against forest degredation.

---

Data:

The data for this project was sourced from the UCI Machine Learning Repository (http://archive.ics.uci.edu/ml/datasets/wilt).

The data was originally collected by the Centre for Environmental Remote Sensing through satellite imagery which was in turn ‘pansharpened’ to obtain spatially and spectrally accurate image measurements.

The total dimensions of the dataset is 4838 observations across 6 variables.

Variable descriptions:

Variable	Type	Description
class	Factor	‘w’ (wilt diseased trees), ‘n’ (all other land cover)
GLCM_Pan	Num	Grey level mean texture (Pan band)
Mean_Green	Num	Mean green value
Mean_Red	Num	Mean red value
Mean_NIR	Num	Mean near infra-red value
SD_Pan	Num	Standard deviation (Pan band)

Known preceding interventions and pre-processing:

In its original form, the data is made up of two separate .csv files which consist of one training set of 4338 observations and one test set of 500 observations.

Undertaken interventions and pre-processing:

• Remove observation 480 from imported test dataset: This observation appeared to be a significant outlier.
  
• Merge test and train datasets into single dataset.
  
• Split the merged dataset into a training set comprising 80% of observations and a test set comprising 20% of observations.

Note: Scaling of observations was not used.

Accordingly, the training dataset consists of 3,870 observations. 3,658 of which are classified as disease free (‘n’) and 212 are classified as diseased (‘w’).

The test dataset consists of 968 observations. 919 of which are classified as disease free (‘n’) and 49 are classified as diseased (‘w’).

---

Methods:

R version

[1] "R version 4.0.2 (2020-06-22)"

Decision Trees

class.rp = rpart(class ~ ., data = wilt_train)

Perform cross-validation and pruning of decision tree

prune.train.matrix$byClass['Specificity']

Random Forests

There is no requirement for cross-validation when using random forests (Breiman, 2001).

Support Vector Machines

svm_class = svm(class~., data=wilt_train)

windows.options(width=10, height=10)

Error in windows.options(width = 10, height = 10) : 
  could not find function "windows.options"

Linear kernel Support Vector Machine

bestmod

Call:
best.tune(method = svm, train.x = class ~ ., data = wilt_train, ranges = list(cost = c(0.001, 
    0.01, 0.1, 1, 10, 100)), kernel = "linear")


Parameters:
   SVM-Type:  C-classification 
 SVM-Kernel:  linear 
       cost:  10 

Number of Support Vectors:  330

Radial kernel Support Vector Machine

set.seed(0)
radtune.out=tune(svm, class~., data=wilt_train, kernel='radial',
                 ranges=list(cost=c(0.001,0.01,0.1,1,10,100),
                             gamma=c(0.01,0.1,1,2,3,4)))
bestmodrad = radtune.out$best.model

---

Results and Discussion:

The table below shows the classification results from the previously highlighted methods. Accuracy refers to the classification accuracy of a method when classifying observations across both of the predictive classes. Specificity refers to the classification accuracy of a method when classifying observations of the class ‘wilt’ only.

From the results obtained through the assessed methods, the radial support vector machine method has recorded the equal highest accuracy (99.07%) and the highest specificity (95.92%) on the test dataset. The random forest method recorded the second highest accuracy (99.07%) and specificity (91.84%) on the test dataset. Decision tree methods also performed comparatively well with accuracy and specificity results of 98.35% and 85.71% respectively for the un-pruned decision tree, and 98.14% and 81.63% respectively for the pruned decision tree. This is an observable reduction in specificity in the pruned decision tree in trade for a relatively minor simplification in the resulting decision tree model (16 terminal nodes versus 13 terminal nodes when pruned). It is clear from analysis of the results that all methods recorded similar accuracy results when each model was applied on the test dataset. As such, it can be stated that no model in particular could be described to have been over-fitted to the training dataset. It is also clear from analysis that all methods tended to perform with less precision when specificity was considered. Assessing the precision in specificity of a method is important when consideration is given to the potential negative consequences of a false-negative wilt disease classification. On this basis, specificity as a determining factor in the overall assessment of the classification accuracy simply cannot be ignored.

---

Conclusions:

In conclusion, this project has assessed and compared decision trees, random forest ensembles for decision trees and support vector machines in the context of identifying diseased trees in the wilt dataset. A radial kernel support vector machine was identified as a highly accurate and specific classifying method for the detection of wilt diseased trees.

---

References:

Breiman, L. (2001). “Random Forests”, https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf

Food and Agriculture Organization of the United Nations Regional Office for Asia and the Pacific. (2010). “Japan Forestry Outlook Study.” In Asia-Pacific Forestry Sector Outlook Study II Working Paper Series, 9–10. Working Paper No. APFSOS II/WP/2010/30.

Johnson, B., Tateishi, R., Hoan, N. (2013). “A hybrid pansharpening approach and multiscale object-based image analysis for mapping diseased pine and oak trees”, International Journal of Remote Sensing, 34 (20), 6969-6982.

Garrett, K., Dendy, S., Frank, E., Rouse, M., Travers, M., (2006) “Climate change effects on plant disease: genomes to ecosystems,” Annual Review of Phytopathology, Vol. 44, pp. 489–509, 2006.

---

Appendices:

Dataset source: http://archive.ics.uci.edu/ml/datasets/wilt